목차
The number of examples in a batch . For instance, if the batch size is 100, then the model processes 100 examples per iteration .
The following are popular batch size strategies:
Stochastic gradient descent , in which the batch size is 1.Full batch, in which the batch size is the number of examples in the entire training set . For instance, if the training set contains a million examples, then the batch size would be a million examples. Full batch is usually an inefficient strategy.
Mini-batch stochastic gradient descent, in which the batch size is usually between 10 and 1000. Mini-batch is usually the most efficient strategy. 목차 관련 글 mini-batch 이 글을 인용: “The batch size of a mini-batch is usually between 10 and 1,000 examples.” Epoch (machine learning) 이 글을 인용: “An epoch represents N/batch size training iterations, where N is the total number of examples.” ML crash course - Linear regression 관련 절: Gradient descent 역전파 “During the forward pass, the system processes a batch of examples to yield prediction(s).” Clipping (machine learning) “Gradient clipping forces gradient values within a designated range during training.” Introduction to Machine Learning 관련 절: Definition of [[Machine learning|ML]] Self-training (machine learning) “Use supervised learning to train a model on the labeled examples.” Machine Learning Systems “Readers will develop the ability to reason about ML system architectures and apply enduring engineering princip…” 경사하강법 “Gradient descent is older--much, much older--than machine learning.” Learning rate “If you set the learning rate too high, gradient descent often has trouble reaching convergence.” Generalization (machine learning) “You train a model on the examples in the training set.” Downsampling (machine learning) “Training on a disproportionately low percentage of over-represented class examples in order to improve model tr…” Minority class (machine learning) “A training set with a million examples sounds impressive.” 지도 학습 “Supervised machine learning is analogous to learning a subject by studying a set of questions and their corresp…” 기계학습 “2024-08-24 - What's Really Going On in Machine Learning?” Training loss “A metric representing a model's loss during a particular training iteration.” Vanishing gradient problem “Increasingly lower gradients result in increasingly smaller changes to the weights on neurons in a DNN, leading…” Parameter (machine learning) “For example, learning rate is a hyperparameter.” 손실 함수 “During training or testing, a mathematical function that calculates the loss on a batch of examples.” Hyperparameter “You could set the learning rate to 0.01 before one training session.”